Design and Analysis of Crowdsourcing Mechanisms

OF THE DISSERTATION Design and Analysis of Crowdsourcing Mechanisms by Chien-Ju Ho Doctor of Philosophy in Computer Science University of California, Los Angeles, 2015 Professor Jennifer Wortman Vaughan, Co-Chair Professor Adnan Youssef Darwiche, Co-Chair Crowdsourcing is a paradigm for utilizing crowd intelligence to help solve problems that computers alone can not yet solve. In recent years, crowdsourcing has gained great success as the Internet makes it easy to engage the crowd to work together. For instance, Wikipedia, the largest encyclopedia in the world, is created with the help of online users. Games with A Purpose, crowdsourcing markets, and online reviewing sites are also platforms that rely on human contributions to accomplish various tasks. However, despite the great success of these platforms, how to better design crowdsourcing mechanisms is still not well understood. Most of the crowdsourcing mechanisms suffer from the prevalence of low quality work generated by humans. In this dissertation, we explore the design and analysis of crowdsourcing mechanisms, with the goal of understanding how to obtain high-quality information from participating users. We approach the design problem from three interleaving directions. First, we study how to assign tasks to workers with suitable skills using techniques from machine learning and online optimization. Second, we explore how to design incentives to motivate users to provide high quality contribution. Finally, we run behavioral experiments to understand how users behave in real systems with the goal of developing more realistic user behavioral models. ii The dissertation of Chien-Ju Ho is approved. Richard E. Korf Mihaela van der Schaar Junghoo Cho Adnan Youssef Darwiche, Committee Co-Chair Jennifer Wortman Vaughan, Committee Co-Chair University of California, Los Angeles 2015 iii TABLE OF CONTENTS